Clustering Inside Classes Improves Performance of Linear Classifiers

This work systematically examines a Clustering Inside Classes (CIC) approach to classification. In CIC, each class is partitioned into subclasses based on cluster analysis. We find that CIC, by extracting local structure and producing compact subclasses, can improve performance of linear classifiers. It is compared against a global classifier on four benchmark datasets, using some of the most popular classification methods such as regularized logistic regression and linear Support Vector Machines. We discuss and empirically analyze the effect of the training set size and the number of clusters per class on the results of the CIC approach. We also examine use of an automated method for selecting the number of clusters for each class.